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This report describes 
deyelop an errorrprone model (EPM) to 
misretoorting of income and family 

government-sponsored school maal benefits.^ (EPM]s are, statistical 
formiiias that produce-^scoring sys^tems us^ to distinguish , / » 
applifcations likely to result in excess benefits from applicaftions 
likelW to be accurate.) A sample of 2,480 cases from the National 
EvaluB^tion of ^the School Nutrition Programs (NESNP) databas^ was used 
in tha^^aiialysis; 1^984 cases were used to develop the model, and 486 
were setaside to test the model after its development. The* 
statistical- procedure used to generate the initial EPM, sequential 
search, made use" of the Automatic^ Interaction Detector program. 
Although analysis of the NESNP data successfully produced an EPM, the 
model had two major deficiencies. Therefore, data fromythe 1981-82 
Income Verification Pilot Pro ject ( IVPP) were u^ed to^continue 
exploring EPM development . The randpm sample consisted of 741 
families from 54 schools participating in Phase 1 of /the IVPP./ 
Results showed that the model developed from the NESNP database was 
validated on in-home aiidit data . The /goal of. producing a simple but^ 
effective EPM also was achieved* The/ resulting scoring profile was 
found to be capable of selecting a subgroup of . applicants who have a 
70' percent probability of rece'^iving excess benefits.^ (RH) 



\ 



ir 



**************************************** 

* Reproductions supplied by lEDRS ^e the best that can be made 
** V from the original document. 

Q ***************************** **********:»f**** ******* ****** 



/ 



( 



U.e. PEI>AnTMENT DP EDUCATION 

NATlONAt INSTITUTE OF EDUCATION 

EDUCATIONAL RESOURCES 1NF0RMAT|0|^ 

CENTER lERIC) 
U Thlti documant ha« been roprocjuced an 
receive^ (rom the puraan or. aruaniialian . 

Minor changes havo boon made to Improve ^ 
reproduction 'quality. 



• Points^f view or opinlomi stated in this docu- 
ment do not necosUj%preaent otlicial NIE 
position or policy. 






;OME VERIFICATION PlCsftROJECT 
(IVPP) 



?■ 



THE DEVELOPMENT or AN ERROR-PRONE MODEL 
FOR THE SCHOOL MEAL PROGRAMS 

Revised August, 1983 ^ 



> Submitted to: 



Jiffic^of.Analysis and Evaluatipn 
Food and Nutrition Service 
' D.S, Department of Agriculture 

« ' Submitted by: r 

Applied Management Sciences, Lac 



This report was prepared.^pfO'rsjjant to Contract No,. 53-3lf98-'153 
with the Office of Analysis and Evaluation, Food and Nutrition 
Services, U.S. Department of Agriculture. However, the content 
do6s not riefcessarily reflect the position. of that agehcyv and no 
official endorsement of these materials should be infer rec|< 



^ECUTIVE SLijVlMARY ^ J 



an 



Vhis , report; Resents efforts made in yl981-82 to* develop 
.error-prone model for USDA's school- meal benefit programs. A 1980 st\ 
by USDA's Office of the Inspector tSeneral and in-h^me audits cd^^^^ 
a part of /the Income Verification VPilot Project reveV|e^.^j:hat . 
^ misreporting on meal benefit* applications resulted in ,the awarcl ofyexceis 
benefits . to a ) substantial^ ' number of ; students./ ^ Therefore, 
appropriate for* the"" Income Verificatipn Pilot Projeci 
Congressionijilly-mandated study of » procedures to prev^t and \detect 
ryiisreporting, to, consiV^r the development of an error prone-model. 

■ . ' • . - ■ ■ - - ■ \ ■ ' , 

Error^rone .models (Eh'Ms) are statistical: formulas that prodiice 
■ • ' * • . 

scoring system^/ which in turn are^Jj^d to distinguish applications 

likely to result in excess- benefits from' those that are likely to .be 

N^accurate. Many other Federal 'benefit programs use eflrror-jprone models to 

reduce fraud and wastfe. v The school meal . environment, however, has 

several features that prevent' an easy adaptation of the EPM systems used 

by other Federal agencies: > ' 

. ^* Applications are processed during a brief period at th^ 

beginning of the school year. Therefore a complex multi-step 
scoring'systlbm would not be ^feasible. 

• Most school food authorities da not use computers to procfess 

applications. Therefore, an EPM would^have to be simple and 
^ easy, to use. • . \ 

•■ .■ . -.' ^ ■ ■ , . V ■ 

V ✓ • Maximum meal benefits per applicant equate to a few hundred ) 
dollars. Therefore,' the model sliould be highfy efficient in 
identifying applications likely, to resiijLt in excess benefits. 
Further, the model should require rrmimai resources for 
implementation. 



/ - ^ . The stHodI meal application contains a lim ; , • - ! f 

variables/ and schoor staff are not likely to HavQ the res'ources^ 

> to access other student records when 'reviewing: the ap^ 

- Therefore^ the EPM scoring system should j^ely exclusively on'the^ ; 
application; ' i 

Given, thesie environmental constraints, * the ,1981-1982 effort sJvto 
develop an>error-prone model were viewed as 'exploratory. The analysis 
leading to model /^development proceeded in an iterative , fashion . The 
first analysis . objective was to determine whether meal , application-type 
data ■woiJid '' be sufficient for producing a model. If the first , goal was 
achieved, the 'seconcl goal wbuld be to , build a modej that would ..be' highly 
effective, yet aimple^ enough to be -feasible in the schooM meal operating 
environment. -y 



are 



The Vresults ^obtained - in responsetv to these analytic objectives 




It is feasible to develop b preliminswj^ error-prone model 
that uses application-type data and is effective in identifying 
applications likely to result in excess benefits, t 



The analysis leading to the model specification suggested two \ 
m.ajor changes to improve the meal Application process: 

• * ' ,■ ' ' ■ 

Applicants should be:required to report income by source/ . 
for each adult. Requesting aggregated income by household 
or individual /tended'to produce omissions. 

Requiring receipt of Tood Stamp benefits on the application 
is ideal for distinguishing truly lligible applicants from _ 
thos^ who misreport. Rec^eipt of Food Stamps was the 
strongest predictbr of^curate reporting. , ^ 

The analysis leading to model development also revealed th^t 
omitted or underreported wa'ge income was the largest factor 
contributing tO' the receipt of excess benefits*. . , \ 

The error-prone model developed and refined in 198T-1982 
produced a scoring wstem that is highly effective, yet simple 
to use. The key predictors of excess benefits in the four-§tep 
scoring system are: ^ • 



receip^of Food Stamps, and . " 4 

reporting tot^l income close to the eligibility cut-off/^ 
point. 



• ' if the error-prone model cdntajnad In thja report Is'used to 
^ ' " select a three percent sample of applicants, cases With excea?i 
•^benefits are four times more likely to be Identified, than would 
- be the case using random sampling. • ' ' . " ; - 

These findings collectively suggest that an error-prone model Is 
an effective and feasible tool for preventing and detecting the award of 
excess meal benefits, in 1982-83 the EPM Will be further tested and 
refined with a more comprehensive data base. 
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This report presents the results of efforts to produce a n^echanlsni" 
••that school distripta might .use to select meal benefit ^ applicants^ vfor ; 
■ verification, in an efficacious manner, "Error-prone models" refapV to 
/ statistical formulas, that us^e application characteristics- or other/ 
accessible data to estimate the likelihood that ^excess benefits are (or" 
would be) received by ' an applicant . A major area of, investigation in the 
^81-82 pha^e of ;the Income Verification Pilot Project (IVPPr was the 
feasibility /of , developing a preliminary ^error-prone models and, Jf 
feasible, the 'development of such a model for testing in 1982-83. The 
~r regiai hder of " th is ' 6h apte r p r es en ts bac kg rou nd j i nf ormat ion~^~con cern i ng 
^ error-pf'one models and their feasilDility in the, school^ meal operating 
• environment. Succeeding chapters present the * 1981-82 efforts in 
^ror-'prone model development an<;I their 'results, . » » 

An Introduction to Error-Prone JModelllng and Implementation b:^ 
Federal Agencies \ ^ 




- As was mentioned above, error-prone models ' are statistical 

^.v^ formulas * used to identify . characteristics or feature^ of -^enefit 

applicants anJ/or recipients that are associatecll with Msreporting. 

'Error-prone models are described non-mathematically ' as error-prone 

profiles; and the -profiles may be .used to. iscreen and ' select applications 

"* ' . " I. ' ' ' - . * 

for verification.^. Thus, error prone models and profiles .are 

. primarily a detection method because they help, to identify a case with 

errors rather thart prevent the applicant from' making >an erj^or. The 

development^ errors-prone models -(EPM) howev.er, often providers insights 

into the nature of .misreporting. This information, can then be used by 
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ppbgpam managaps to eh^ng^ th^^ application dooumeatv .lthe /applleatl^jn 
proGBss, ,an.d/op' verlfleiition proQaclHrea sb that errops can bs^ prlventad. 
The EPM clavelopment efforts In IVPP f^foclucicl' both types of re§ulWl ' ■ 

T.he Mae of an eppop-prpne mgclal^ offsps aevepaP advantages, First, 
It ppovlclas an objeotlve baijis \fop selecting applications for 
vaplflcatlon. In contrnat' to experlenti'al or Intuitive aalectlon based 
on auaplclona, i»n arrop-prone model offary a non-biased approach to the- 
idontlflcatlon ^0^ appllcatlona' that 'warrant \further •examination. Second, 
aelocting only the cqae$ that ope moat likely to contain arrora allowa 
program managers to gheatly Inpraaae the e;ffactlvanQ38 ,of verification 
. *actlvltlea, ' In. contrast to random sampling, of ^applicants,, an errofx'prone 
model permits, limited resources to be applied to\ those applications whore 
the ;pa^-off in errors found and savings^ achieved are "likely to b^ 
greatest. .Further', orror-prrohe ^model-based seicictlon excludes from 
verification - those applicants who have very \jow probabilities 'of 
misreporting,^ thus reducing applicant burden andl preventing' .wasted 
resources. ^ ' \ ' ) ^ 

Many Federal agpncles currently use error-prone \models as Ja tool 

' to sufspor^ management of their programs! AmiJng them^ are the 

Department of Agriculture's Food Stamp ' Program, the^ Department of 

Education's' Pell Grant (student gPants for. higher education! Prdgram, "the 

Social Security Administration's . Supplemental ^ecurity Income Program, 

and the Internal .Revenue Service's Federal Income Tax Progra^n. A brief 

' synopsis of the rOle of EP'Ms in these other' Federal agencies follows\ , 

■■■■■ ■ ■ ■ ■ ■ ■ ■ ' '\- 

FNS has funded research projects in error-prone [tTOdeiing/for the 

Food Stamp Program. In North Carolina, an error-prone model is used • to 

provij^e .a ; basis for targeting the work of personnel involved in 

verification . of applications. In New Mexico, 'an error-prone model has 

been developed to identify the most sighificant .types of errors and t\}e\r 

causes. New Mexico is* how examining -cost-effective methods fpr 

correcting these *types of errors. 'An error-prone model developed by 

Texas is being used to identify low-risk applicants who can be given 

extended certification, thereby permitting r:eallocation^ of administrative 

resources to the applications that are more likely to 'contain errors. 



Th§ .SoRial Seegpity Adminietp^tjtin m?4§ appQi^-prpna mociel in 
manapin^ the Supplempntal pecupity Income '(SSI) Ppogp^m. 'The' mofiel 
spe^lf^QS th© avepag© clollaiN eppop fop dlffepsnl; types of benefioispies • 
anel the* peaiilti spp used to establish pecJetepmhiatlon ppofiedi/^eg fop eseh 
beneflclaiYf baaed ■ oi? the expeet§V^ clollaP' loss, Ifi oasai where lapa§_ 
loflsas ap.e anticipated becaiiae of the high ppobability of errop, 
reidetermlnations are Bondnoted mope fpeqiiently 'and involve mope detailed 
procesalng, The jnodel Is daaigned to dii;eot pedetepminatlon el'fopts to 
th<i typoa of benefit casoli whopa tlie tjxponse of pp.oceaajng -is most lil<0ly 
• to be offset b^' aavlnfla peaultina from the detection of peilatlvely costly 
eprora, . ■ ' ' . ' „ , ' 

'The LJ.S. 'Department of Ediioatiop, Office, of Student i'lnanciai 
AaslstancQ uaoa an orror-prono mocJol in manaoina itfi, i'ell Grants 
Program. This modol is usad to prevent tliQ. miaaiiosia^tion of Fodoral 
student ' financial aid funds.- While procogtalhg the applicationa, thoso 
that match a„pr.ofl!e__foTL„.j.„bj0^^^ 

for verification. These applicants pre askod to provide documonta*. to 
support Information shown on their application forms. Selection of 
applications Is performed by computer after oljgibility Is determined and 
verification is performed by local college financial •aid administrators. 
No benefits are awarded until application data are verified or 
corrected. 

In contrast to the pre-award review per?ormed by the Office of 
Student Financial Assistance, the Internal Revenue Service uses 

error-prone modeling to review tax returns after initial processing. 

J. 

Once returns have been processed and refunds haye been made, IRS selects 
for'audit tax returns matching a profile suggestive of a high probability 
of error. Taxpayers; are then contacted and undergo an audit. If errors 
■pare found, the taxpayer makes payment, plus interest to the IRS. Thus, 
alt^^^ IRS performs its audits after return processing, the penalty and 
payment requirements equate their system to a pre-award detectio^ 
system. J ' " 

As* is evident from this brief discussion. Federal agencies use 
error-prone models for a variety of purposes: 

' " ' 9 ■ ■ ' 



. •• ' Tq gain' a better Mncl§r§t^n'Jine flf ^mr 4QMrcas ^o'that 

' ■ ■ ■ " ' 

• ' To target adminNtrative pa§QMP6i§ m that ^avintjf foi- , 

. srrop' ditPRtian §MPpasVthe''aKp§n§§§ of p§r'f9/niiiVE) 

verlfieation, ' ' , . 

«, To reelwGe apfklioant bwrelfilfi by iliminatina from ' ♦ 
\^v§rifiQation ifforti ^ppliGfttions ^iHi/or bani*fifiiat'iss iiksly 
{101^ to b§ in error, 

, These .eKamplis' also reppi§§nt. a v^Heiy of opepflHonal 
♦invlronmenU, Pop ' linstanee,. thi^ l^ill Qrant Program ii adrntnisterafi 
oentpfllly by the Petldrrtl gaveriimant St'ufl^ntii apply dlractly .to the 
cjovernment and appHeat^ons ara 0rno§s««d by t^oritraoior In « eantral 
^nd latelllta sites. JRMs ' are- applied • by computer n a part (^f 
application prooisatn'g,' th^n \qq^\^ ooll|^gft fin^HGi^l m\ 

oiher' programs, tha/'tiUI:^ pUiyi a rol§ or the appllq^itlpn P''«f^^»^ octiuri.. 
at tho l9c«l.lavaL For Jn»Unca, in proprAim* «uah tfisi AFDC ond Fogd 

stata agonclpji often ara rasoohslbia for developing and updatiha th« l;PM, 
and. ovorsdaing Its uso by local iiganclo^V Proflla^ ar^ providmJ to local 
offices wtior<i caseworkora manually scnJon applications or, if thfj 
application process is coipputerlyod/ the. applications are scraoned by 
computer. In programs such h% Food Stamps,- AFDC and SSI,, all 
applications are verified^- before benefits are awarded; and 
error-prone models are used'to; 

• Identify cases for more frequent or less frequent ^ 
redeterminations, 

/ • Identify cases likely to require more extensive or less 

extensive verification . ► 

None of * the programs highlighted here operate exactly as * the 
school meal, programs do. The" school meal programs, are similar to Pell 
Grants in that the' local administrative staff are not paid directly for 
helping the Federal government administer the program. Jhis occurs 
because the programs themselves are thought by some to produce an income 
for the educational institution. In the case of Pell Grants, the program 
makes higher education affordable to students who otherwise would not be 
able to attend, thus supplying more income from tuition fees, etc. This 



mmm^ is rie^^ipa ^nd' maintain f§6tl!tt§§; and tq pr^^vida 

pr^qr§m§.i the Federal r^ifiibur^^m^^nt f^^r m§^iU ,^ad ^tudpnt p.^vfiiints for 
miiU' provifis s ' signifiesnt li¥§l fwnfj§ rt§edivi tf? tuppert tfie 

f»s*iiait^nt'P . proai'f<m!j WH^ AFPC^.. S^l. sihJ Pwd Stsmp^ da not pradMRH 
jfiPflmti for- tim lapal Ariminii*{t*rina fitlsiifiV: Insitfjad, Uifi ral# th«* 
looal ^QtiiGy is to pro0J4,^ f^pplifi^Uons and dist)iiri?(ii mon§tfiry or 

Th« Pt^ll rti'snt program U f^lso simikr to th^i mesi protji'fimii in 
th4i tl^e«*Hi«ijari^v ban#flt detarmin^tiona acciir in the liugiiinina of the 
m\\QQ\ ya^r. in r.Mnl;i>iil, t)tl>tii' f^^dwrfil ^)Bnefit 'piagpams' nornisliy 
prou^ai ^pplicstion^ At s rajitiveiy uniform rate throuyhout the ymr-. 

Althouqti moat siimllsir to th# Ptsill fu'-ifU i*royrtim/, tli^ siolVotil ois^l 
pnigrtims differ from P<$ll .tirf^nta in on« important , r«ia^f,t. In,th«* stjhepi 
phigrams applicatjon prwjssaif^jj , and ejlQlbil^iry detarmination* are' 
p0rform«d tiy tht» ^jganGy that Is dlso rtisfjanaltjltt for dialuirsiliiu 'tlm 
m«<^l t>(*iH»fits, In the Pall Grant Program, Ipa^l agnnnMa arH r«g|joiiHlblo 
only for disburauinfunti; th^ Paiiorcil ,yov«rnmflnt U rasponiibi© for 
, rovlowtfig iincl iipprovlrVy tho .ippllc^itloh. thin ranpact thd school moal 
program* iiro moro alrniUir to AFDC and Food Sttimp?i bocmno loc<tl $t«iff 
h.uu1lo appliOiUlon procoH?ilnrn, ^ oliglblllty dotarmlnntlons, and 
dinbursornanta. ^ ' 

ThoHci slitvllarltioa And diffaruncrnk domonntriito Uu^t: no othor 
Fodoral progrnm pro*i/ldos a procUo aH.'unF>To t)f (:PM oppllciitlon that could 
be tronsforrod to tho school meal programs. A summtiry of issues 
associated with adapting any EPM systdm. for the sgIiooI meal environment 
follows. ' i 

Applying Error-Prone. Models in the School Meal Operating Enviroment 

The use of EPMs in the school meal programs is being considered by 
IVPP because there is a substantial amouht of error^ in the meaf 
application process 'and as a consequence public funds are being used to 
provide benefits that exceed the true eligibility of some- applicarits. 

12 



the errpr rate is documented in earlier IVPP H^pbrts^ and in 
previaus research by USDA's Office o^ the Inspector .General. Therefore> 
it is incumbent' up.on program administrators to take corrective action, 
and it is' advisable that the corrective action be as cost efficient as 
poi^ible. Art erro^-prone model strategy is a^Wsely Candidate to support 
the proper ma\agement of the programs. However,, the scTiool meal programs 
ftave several 'unique features that, afffect the feasibility of 
implementation: 

• Families are required to reapply for meal benefits annually. • ^ 

Because schodls traditionally complete administrative papenwork 
at^^the beginning of^ the school year, virtually all meil \ 
applications are processed before October of each school year. 
This places a burden on determining officials for quick review 
and processing. In many*oth.er Federal, assistance programs 
eligibility determinatibp and verification occur(!hrbughout the 
^H. ' year. Therefore, the.l^rden of conducting, verifications based 

on an error-prone model is spread across the year rather than 
occurring all pt one time. This concentrated workload problem 
is unique to the school meal programs an.d the Pell Grant ^ 
Program. 

• Traditionally, the school meal programs have accepted 

- self- reported, unverified information about household $rize and 
income' and have base<^ eligibility determination on these' 
reports. In contrast, many other Federal assistance programs . 
require documentation of income to support the eligibility 
determination process. The imposition of an error-prone model 
and additional verification requirements may meet resistance 
from both parents and SFA officials. 

• The enabling legislation mandates that the application 

form foi; participation in "the school meal programs must be easy 
to complete. The amount of information that can be requested 
is, therefore, limited and the form must be ea^y to understand. 
The need to use an uncomplicated application affects flexibility 
to add screening items that would support identification of 
potential misreporters. 

< The school meal programs have a relatively low per student 
dollaf benefit*rate--approximately $200 per student annually. 
Efforts to reduce rrjisreporting will not be cost-effective unless 
the amount saved through prevention and detection of ^ 
misreporting is great enough to offset the additional ^ 
administrative costs associated with verification. 
Cost-effectivenass may be more readily achieved in other Federal, 
assistance programs where ben^efit levels are higher. 

Applications for meal benefits are processed by SFA and/or 
other school officials who have many other responsibilities and 
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limited time to devote to th^ eligibility determ , , 

process- jmplementing an erpor-prone model would place an r 
additional burden on these personnel. In contrast, some other 
Federal assistance programs assign personnel to application 
^processing, eligibility determination, and verification as a 
major aspect of their job^. This feature has implications for 
the cost; and feasibility of implementing additional verification 
requirements based on the results df an error-prone model. 

School food authorities do not receive any Federal or 
state funding to conduct eligibility determination and 
verification efforts. Ah error-prone model may introduce 
additrohal^taff requirements that are not currently funded. 
SFA officials belieye that they may not be able to perform 
verification unless th^y receive additional resources. 

There is no provision in the law that would permit 
recovering funds from individuals who received free or 
reduced-price meals for which they were not eligible. Other 
programs have such provisions or are able to adjust behefits to 
reflect the results of verification. Consequently, error-prone 
models will have the greatest pay-off if they are used early in 
the school year before substantial benefits have been awarded. 



These features create some major challen^s not only for the 
development of ' errdr-p'hone models, but also for virtually any form of 
corrective action. Given these circumstances, it was decided that the 
investigation of error-|j>rone models in IVPP would be most sensitive to 
tbe following issues: .^^ ; 

• Because the maximum amount of benefits that could be 
overawarded is so^small, the model must be able to target in on 
highly error-proae groups that would make verification cost 
effective ► 

• Because of limited SFA resources and the lack of 

widespread computerized processing systems, the model should be ^ 

quick and easy to implement in a manual processing 

environment. 

• Because legislation mandates a simple applicatioli, the 
model should ideally rely'on existing apprlication data. 

These issues dictated an iterative ' process for . model development 
that responded to the following questions. 

• First, could an effective model actually be developed with 
the limited data currently available and potentially available 
on the application? That is, could a model be developed at all, 
and if so, would it be effective in identifying groups of 
applicants with an extrertiely high or extremely low error 



■ rate?. V ; 

• Second,? if it was possible to develop a moc^el , could one 
be both effec^ve in detecting applications with high error 
rates 'and be' non-burdensome to use? 

These questions are addressed jn Chapters 2 and 3. ?. 
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END NOTES 



^For the purpose of simplicity, the terms error/prone models or EPMs/ 
refer to both models and profiles. 

*Pre-award verification methods used by other agencies include: 
requiring documentation with the application; collateral contact with ' - 
employers, banks and other Ibdgl agencies; and inter-county and ; 
intra-state tape matches or data exchanges with employment security 
wage files, motor vehicle files, child support and enforcement records, etc 

^The I VPP School Year 1981-82 In-Home Audit findings (Applied* 
Mianagement Sciences, May 1983) found that about 17 percent of the 
. applicahts for school meals had application errors that produced an 
erroneous eligibility determination. A 28 percent error rate was found 

by. ' -.y^ 
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DEVELOPMENT OF A PR.ELIMINARY ERROR-PRONE'MODEL 



5 c>eqir>nin< 



At the' Deginning of the (^'VPP i(t 1981, FNS possessed two nationalty* ; 
representative data bases thaf potentially coufd be^/use^^to attempt to 
develop an error-prone model. ^ N^her data bas.e was\^»eal/ however-, 
because neither contained bdth verifFed eligibility data ^atHL^^plicatfbn 
data.^ The two available data bases were: ' | ' 

• An OIGlaudit d^ta ba^ containing information about 766 meal ' 

program participants collected in May of 1980, 

• The National Evaluation of the' School Nutrition Programs 

(NESNB) data base containing Information about 7V628 meal 
program j^ra^s^cipants collected in the Spring of 1981, ^ 

■■^ " ■■■■ ■ - ■ y . ■ • 

The primary advantage of the OIG data ^^ase was that it possessepi 
eligibility, errors detected by audits. For thie ypurfSose of EPM 
development, however, it was. limited because it, was ^ a relatively sm^ 
data, base and contained few variables from the, application that c6uld be 
used as predictors. For instance, it did not contain^ children's grade o^ 
family income sourcies. Further, the data' were older than the NESNP data^ 



and did not reflect program changes that occurred between '»1980 and 1981. 
This fact would reduce the gerieralizability of the EPM results. 



The NES^NP data base also had a major limitation relative to EPM/ 
development. The criterion measure (aw^rd of excess benefrts) was based 
on personal Tnterviews,. not audits, H^ever, the recency of the data, 
the large data file, and the availability of numerous variables /to be 
tested as predictors led to the selection of NESNP for the development of 
a preliminary EPM. 

Because NESNP was a comprehensive evaluation of the school, meal 
programs, all data available from NESNP were not. relevant to EPM 
development. Therefore, a sample of 2,480 cases that met the following 
conditions were considered for analysis: ^ , 

' . ■ .ft 




• The- child was reported,^<^he parerits as^^p^^^ or ■ 
. ' reduced-price breakfa^s^: lunchfes. ' (NESNP also incltided 

children who/were ineligible and children who received milk 
■ ^ ■, only.) , . ' . ■ ••■ '- ^ V/ \' ■ ' ' - : 

^ • ' Adequate data were a>?ailable 16 permit an eligibility 

detlrm'ination ta be made. (Since NESNP was based on intervie 
. with parents, the application was not available an^ eligibility 

had to be computed from the interview data J 

• The computed eligibility either matched or exceeded the 

parent's report of eligibility. (Since application-based ^ r** 
^ eligibility was- not available, eligibility was computed from^ the 
. income and household information ;reported during the interview. 
Only those cases where the reported eligibility match^or ' 
exceeded computed eligibility were used in analysis.; We were 
not interested in cases where the computed eligibility was less . 
than reported eligibility^ because we were^^ interested^ 
in identifying casfes' of excess benefits, not insufficient 
(J^nefits.) ; • : ^ ^ . : 

9^" the sample of 2,480 cas'^esi/ 1 ,984 cases were used to develop the 



was 



^model artd; 486 Vases were set aside to test . the model , after it 

developed". > Thus) results based on the sample of 1,948 should>r^^^ 
' the sample of 496 iKjEhe model is generalizabl^ ' V 

The Predictive Variables Selected 'f6r Testing f _ « 

There are two' reasons why a vari^b^^ m^y ^ficiently predict 
misreporting and, excess ' benefits, FirstC (the- variable may be r^ted ,t{^ 
characteristics of the individual that ■ mi^, niake him ' or . her careless of 
dishonest or that may make jt difficult for the applicant to /^deterriiine 
his or her own income. Such a characteristic could- be self- employment; 
" foV example,^ because self-employed individuals are likely * to haVe 
irregular: 'and unpredictable incomes. Second, ;the, variable may be * related 
to a person or' ftfmify having in income iabove the income 'eligibility 



cut-off. Owning a home might be such an indicator. 



Predi^tofs were selected from a"^o"g *h^^ny^^present ^ 
data base if they met one of the following crite^^^ ^ ' ' - 



NES^vlf^'^ 



/!/• The predictor could be easily accessed from the application 
^ or school recordsVand could/ therefore/ be used to screen 
applications on the^ basis of an EPM> - ^ 

• An applicant would be unlikely to distort the. information 



erIc • ' ■ 



-.mm- - 

Ani-r^xample of a predictor that meets the first criterion would be 
nunjpier of siblings in school, whidi ' coulcl be obtained . from . the 
amplication (in some instances) or from school records. A second example ^ 
iSOd'catipn of. home (urban, sjuburban, small or rural),, which could 

iDeiM^termined from the . address or location' of 'the school,' Parents* 



^^t^i^d^ (or at least whether they had ^a non-iarm, labor occupation). 

.fl^iUiy be determined ^rom other records- independent of Mlie 

» |ll|^li cation (e.g.., parent emergency contact information maintained by tne 

#Wcipo6ls). Clearly, some of these variables would . .|?e easier\ to access 

' second criterion (i^e. , variables jiktely v|to ^ be reported 

. accurately because ^ their connection to eligibiyiy^ determination / is npx' 
<^ immediately obvious) somewhat less precise. When " asked, jnost^ people 



,w|ir describe their occupation accurately, though! they rnj^jJkt not 
Jm aV cont^xt where they though cpuld affect their eligibility. The 
same may be true pf home pwnership. Variables such as participation in 
the I^MDod Stamp Program or other assistance programs are. unlikely to be 
distorted, hov^ever, ' because iof .the applicants* perceptions that their 
answers could ^eTverified eiasily. A . . ^\ 

Income was not used as a pYedictor (except for reoeipt or 
nonreceipt of public .assistance) because it is precisely this variable/ 
that the""^"p^^^ mqst li}<ely to misrepprt. ' Income was, however, 

used J in exploratory analysis to- obtain a better ' unders'tapding of the 
nature of misreporting. V Correlations bjetween the presence and absence of : 
different sou'rces of income and misreporting were calculated to determine 
whether any relationships existed/ The analysis reveaM that number of 
earnersV'w.as^ highly,, correlated with misreporting. Number of earners was 
not used, as a predictor variable, ho\^ever, * because this information was 

'hot requested on the 1981-82 application. As the next chapter indicates, 
this information can be helpful for making policy recommendations about 

t the format of the application. . , ^' V / 

One addilTohal . consideration in the 'selection of predictors is 
"^ce validity . • ,F^ce^ validity is^^ the intuitive judgment; that a variable 
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^ should be : indicative of misreporting. Some Variabjes vma.y correlate^^^ w 
mjyeportihg tlirough chance or tlirougli a con^plex ;,hetw<^k of causes^ and 
^ffebts. .Unless tlie Vafriabld^ can, bfe - credible |is|V^^^ independent 
of './its ^empirical properties, it wobid be ^IJrfipult to implemeht tlje. 
model. - For ) 'example, • '^g>ade o^ stugltit may^- correlate sigtiificantly . with 
' misreporting by. chance, 'because,^ data were.|cpllecteci only from students in 
I •s'deftt'ed' grades; or .biecausey there,/.'l§- -^a.,^ai;; relat)onshl^' between ^gpa^ie 



level -ahd misreporting. Because the yai^^^ have , strong face 

validity, it; might not be inclu6le^1n ^^n'|^|ror*^^^^ 

A complete 'Vist " of ■ .^'i'alijiles^ used and their results are ."^ 
'- Appendix- A. . EPM are operationally ; 




led in^Appen< 



The 4iSMMi»^^ 
3vs> some rheasarteftW' ei^ Depend 



error-proh'e mpdel deveJopmen^ 



IS 



• 'alfeys> som^' me^stjirlsi^ ^Jiependir^g on the available data a\d^ the 

hee^ofVtlie agfn(s|?^;|ha^ will use . the model,^ errors can be defined as • 



disccepancjes ' betW|%p;-^^^^^^ inconie,v/ between" correct 

eligibiMty %nd^^^^^^^^^^^^ cfitermin^'ti erroneously, ^ and/pij ; between 

' benefitramounts;^Mivarded and the amount that shoul^ \have been , awarded. ^ 
based' on .accuVt/e . applicatioir-.data.. For .the 'school ^rtieal -programs, error ^ 
is defined.^asV^^ny instance in which" a' child appears- to, have received f ' 
benefits in excess of ' those to which he/sVe i,§ entitled^ iricl-uding: ' 

• Receiving a free oi; reduced-price. meaLwhen he/she is not 
*' entitlfed to either benefit; or , ' j 

* Receiving a free me&l wheifi he/she is entitled only .to a 
- reduced-'^rice meal. f . • ' ■ 

, The determination of\ what the child received w^s based on the ' 
^ famity's^ response in the NESNP survey, unless there were inconsistencies 
' within the survey data. In cases with inconsistencies, the follovying 
rules w^e observeci: K ' / V 




jf a respondent did not report receiving irMii^ 
'reported receiving free breakfast, thQ tar^t) child in* the 
family was classifisid as receiving free meals; and ^ " 
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If .a respondent peport^d receivrng reduced-price' luncfies .v ; 

but reported payirig/more than the Fe^ ' 

the'ltime) for the lunchV the target chfl in the family was ; . - J . 

excluded from the feampley ^ ■ ' , - ^ ^ 



T^he/ formula used to ^a^roxiiflate^ the free or reduced-price mear 
staM^tov which a ^respondent was entitled involved comparing reported 

Jme ^he eligibility scales. Adjustments were made to allow* fo/* 
sitvatipns in which ; hous^hold^' may .have been given a^ " hardship 
deduction,^ ' because- . of high housing ^ expenses -on . other unqsual 
Bj<penses. - If the applicant received more benefits than he/she Jwas^ 
entitled to even after a hardship adjustment was taken into account, the 
application was classified as involving error. : . .^ j.: 

Because the NESNP data base did not identify whether an applicant 
rec^ved. a hardship deduction/ the proiStoW^e used • provided a gepeApu^ 



interpretattdn of eligibility . I n' other words,, a child determined/* to be^ 
eligible by the hardship formula may or may/ nfot have been eligible,- but 'a 
child determined to be ineligible could not have been eligible uncfer any 
interpretation of the regulations. A .correction of this type will;, not y{5e 
' needed in the future because the enabling legislation no longer pieprnlts 
^'granting hardship deductiorl^s. 

Development of the EPM ^ . 

^^'fhe statistical procedure used to generate the/ initial error-^prone 
.model was sequential search;, using the Automatic interaction Detector 
(AID) program developed "by the' Institute for Social Research ^ ^t the 
Universityi of Michigan . Appendix B discusses the specific parameters. 
' used jn the analysis. AID has been used wjth considerable success by 
developers of £PMs for a number of agencies. 

AID uses a hierarchical scheme that divides' groups of ' cases in 
• such a way that their" differences are maximized. In simple terms, AID 
f picks a,s its 'first predictor variable, the one that can ^ break the total 
sample into two grpups--one that has a large proportion of cases with 
errors, and a second that has a.small proportion of cases with errors. ■.. ^ 

Each of the two . groups As split again, so that the difference 
between the proportions is maximized, and the p^^ocess continues iFr^ari 
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Iterative fashion, uptil a subsample can no^ longer be split meaningfully 
be.cause, for example, the number of* cases in eacti of the- tw6 subsan^es 
is 'very small .or differences between the two- subgroijpX'a smaNy 
Any subsarfiple that cannot be split further becomes one. of the groups 
defined' by the model. 

Validation of the Model ' , 

The , results of the preliminary effort were valid9t|iJ'- using the 
replication sample. As noted above, , the' rpplicatio'n sample was a subset 
of the NESNP data baSe that was originally set ^ aside to provide a 
separate data set for testing the predictive Value of the preliminary 
groupings. Splits thai^ vyere not validated by the replicarlion sample were 
a'pparently due to chance. The variables that produced chance splits were 
'dropped as a result of the findings from the replication sample and neyv 
splits that did not include these variables were created. s 

Results 9 

— [ — ' — '. — ^ ^ ■ . ■ * .• 

' The EPM development using NESNP data had three results: 
mis reporting rates estimated from the NESNP data, variables associated 
with high error rates, and the model itself. . 

Mis reporting Ra'tes : Exhibit 1, presents findings on tjie, extent of 
misreporting in the NESNP sample of liouseholds. It indicates that 
approximately 80 percent of . those families reporting receipt of free 
meals (breakfast and lunch) were indeed, eligible for these benefits. 
The'error rate was higher among those reporting receipt of reduced-price 
lunches; only, 49i.6 )>ercent of this group was receiving the 
• app.ropriateleveL of benefits. Because many more respondents reported 
receiving, free meals than reduced-price meals, the total error rate was 
approximately 22 percent. Tfie analysis reveals both cases of children 
receiving! higher and lower benefits than those tO; which they are 
entitled; Imuph of the error in the reduced-price category is associated 
with children paying for meals when they appear to be eligible for fbee 
mesls-/ , 



■ ■. ■■■ . ■ • ■ ■ ■■ „•••..,.. . • • ■ . ..' ■ ■, 

EXHIBIT 1:: ERROR RATES IN A\?ARD OF BENEFITS FOR/SCHOOL NUTRITldN; . 

' v . PROGRAMS-/- ■ ■ '. 

ELIGIBILITY STATUS AS! DETERMINED 
■ ' BY. SCHOOL-OFFICIALS ' 

CORRECT 



ELIGIBILITY 
•'STATUS 


Receiving 
Free Lunches 


Recetving . 
Reduced-Price 
.Lunches ■ 


Receiving 
Free Breakfast 


Receiving appropriate 
level of benefitS'..-4|fc • 


■78.7% 


' 49.6% . 


.81.5% 


ShouVd pay full ijrice 


; 5.5 • 


■19.5 


18.5 


Should pay reduced price 


15.8^ 


NA 


: NA 


Should receive free f^ieals 


NA 

... i. 


30.9 


NA 

: \ 



.- Based on an Analysis of the NESNP household .survey of parents. 

' ■ ' ^ . ■ ^- - . ■ ■ - 
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It should be , noted that these findings are : based on reports by 
parents abo^t both their income and their child's ^eligibility status. 

, Error rates jwere estimated without reference tb the application or data 
on actual ^benefit status. Assumptions about the reasons for 
discrepancieij between income level and eligibility status were made on 

^the basis of^ecision rules noted earlier in this chapter. The findings 
suggest that personal interviews with parents, in which detailed income 
information was elicited, successfully obtained information about more 
income that was reported on the application^. . 

The error rate found, in * this analysis is somewhat Tower than the 
rate of error found in previous OIG studies/ This may have occurred * 
because reporting accuracy increased. A morei likely explanation^,, however, 
• is that the NESNP data base -permitted a less restrictive estimate of 
error than did the OIG study. The OIG study used verifiedMnformation , 
In this analysis errors were based on discrepancies in self-reported 
\ eligibility and eligibility based on self reported income and household 
A data. However, the error rates estimated from NESMP are consistent with 
^the finding of extensive misreporting in the school meal programs. ^ 

\ Va riables Associated with High Error Rates : \ The ^ analysis 
identified variables that appear to be associated with a high rate of 
errors in eligibility status and vari'ables that seem to be associated 
with accuracy inVeligibility status. Exhibit 2 presents a listing of the 

__jyi'_^a.dables_r-dated,-:to^.^^.-^^ Because the results are based on a 

bivariate analysis that continually splits the^ sample into two groups 
(one with a higher proportion of errors and one with a lower proportion 
errors) the variables presented in the two categories are complements of^ 
each other. , 

Most of the variables that are related to a high error rate are 
indicators of being relatively well-off financially. For example, a 
higher proportion of error is. found among households: (1) with income 
from dividends or interest; (2) with multiple wage earners; or (3) who 
own their own homes. These factors are indicators that family income is 
likely to . be above the eligibility limits , and also appear to be 
surrogates for carelessness or dishonesty in reporting income. 



High Error Croup 



Error Rate 



Percent " 
of Sample 



Low Error Group 



Presehce of dividends or 51,9. 
or Interest ' 

Two or more wage earners . ^I.S 

. Do not receive Food Stamps 35,5 

Self-employed 34.1 

Presence of 'male acHilt in 32.4 

non-farm labor force ^ 

Male adult has some high 32.1 
school education * 

femalp adult absent or in 32»2 
non-farm labor force 

No siblings in school or at 31 .'1 
- least one receiving 
CD ^ lower meal benefits 

Own home 28.5 

; Do not receive public - " 28.1 
assistance ^ ' 

At fiiost one sibling , 27.9 
attends school • ' 

■J 

Hale adult present 27.5 

Female adult Is high 27.4 
' school graduate ' 



2.7 , No dividends or interest present 

34,0 One or no wage earners 

53.6 ^ . / Receive Food Stamps 

4.1 . Not self-empioyed 

49^6, ■ Ha(e adult absent, farming 

or not In labor force ^ 

42.4 ' No male adult or male less 

: than a high school education 

46.5 : Female adul t, farms or Is not 

pn labor force 

21.0 : Some siblings in school 

and all ^ecefve at least 
some benefits 

■ . ■ ', . ■ , • ■' ' . 

52.1 ■ Do not own home 

71.5 " Receive public assistance 



43.2 Hore than one sibling 

attends school , ^ 

63.2 Male adult absent 

47.8 No female adult or female has* 

less than a high school 
■ education 



Indicators of lower income, such as; receipt of public assistance 
payments or Food Stamps and the absenj:e of a wage earner are associated 
with lower error rates. These measures are indicators that the hbuseholcj 
probably does have a relatively low income and is eligible to participate 
in schooj meal programs. 

Exhibit*^ 2 also shows the proportion of the sample that ha$ each 
characteristic. For example/ the likelihood of error in eligibility 
status is 51.9 percent for households with dividend or interest income 
but only 2.7 percent of the sample housfeliolds have this type of income. 
When selecting variables for an error-prone [nodel, both error rate and 
prpportion of population must be considered because the most useful 
variables are those that are both characteristic of a high proportion of 
the^population and associated with a high probability of error^ 

The most important findings are presented below. 

• Household size/alone, is not a useful variable-- what is - 

important is knowing whether the. household contains a large 
number of children or adults. A larger number of children 
attending school tends to be associated with program eligibility 
and low misreporting whereas a larger number of adults in the 
household tends to be associated with higher income and higher 
misrepbrtirig. 

• Source of income is an important variable because self- 

empjoyment and employment in the nonfarm labor force tends to be 
associated with higher incomes and higher error rates whereas 
income from farming or transfer payments /tends to be associated 
with a lower rate of misreporting. 

• Food^Stamp status is a valuable variable because receipt 
of Food Stamps tends to be associated with lower rates of 
misreporting. 

• Variables that seem to' be especially useful because they 

arevassociated with a high rate of error and are found in a 
large proportion of the population are: household not 
participating in the WIC program; presence of a male adult in 
the household; living in a single family dwelling; household not 
receiving public assistance; and household not receiving Food 
Stamps. 

In summary, the' analysis which ^ prefaced the model development 
suggests that variables which are surrogates for income above poverty 
(e.g., home ownership, living in a single family dwelling, higher levels 
of educational attainment, income from wages, small family size, etc.) 
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tend to be* associated with ineligibility and error. Program eligibility 
and accurate reporting tend to be related to surrogates for low inconie 
Cevg., participation in public assistance programs, large family size, 
little' or no income from wages, sin^e female head of household,, 
unemployed male/ etc.). 

The Groups Generated by the Error-Prone Model ; As was mentioned 
earlier, ' all of the variables that were analyzed were not considered in- 
the development of the model. Instead, only those variables that met the 
following conditions were used: 

/I) They should be available bn the application or easily accisssible 
*^ I so that they could be usfed by schpol personnel in applying the 
model.* ' 

2) The accuracy of the variables!available to school personnel 
should he the same as the accuracy achieved in NESNP. For 
instance, becauise income- amount tends to be misreported. on the 
application, and may not be misreported in the NESNP interview, 
income amount from NESNP would not be a useful variable for the 
EPM. / . 

The ^ initial error-prone model divided the sample into two 
groUps--one with a higher proportion of errors and ' one with a lower 
proportion of errors. A total of 14 mutually exclusive groupings were 
generated in the initidi model using variables that might be available to 
determining offici^^t and were characteristic of a substantial proportion 
of the sample. These variables were selected because they would "produce 
the most efficient scheme for locating misreporters. 

Exhibit 3 is a tree-diagram that shows the subgroups that were 
firoduced by the initial error prone model. The statistic shown in 
parentheses js the percent of the sample possessing that combination of 
characteristics. The E.R. statistic is the* error rate found among 
individuals in that grouping. For example. Group B is recipients who do 
not receive Food Stamps, where the female adult is absent or in the 
nonfarm labor force, tl^e male adult is in the nonfarm labor force, the 
household is not near a large city, and there are no eligible siblings or 
"the . siblings pay for school meals. This group constitutes 2.5 percent of 
the sample and the error rate for the group is almost 70 percent. 

A second summary of the groups is presented in tabular form in 
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Exhibits 4 and 5. Groups shown Jrv- Exhibit 4 have above average error 
rates; groups shown In Exhibit 5 have below average error rates. If the 
error-prone model were used to select suspicious applications, groups in 

Exhibit s would be selected for further verification of income data, 

' . ■* " ■ • ■ . ■ 

Conclusions about the Preliminary Error-Prone Model Developed from 

NESNP Data ~~ ^ 

This analysis produced important conclusions. 

• First, it was possible to develop an error-prone model for 
the free and reduced-price 'school meal programs, 

• Second, use of this error-^prone model in selecting 

applications for verificatipn would probably pr^^ 
^ savings over random selection of applications. 

• Third, the analysis leading to the development of the 

model suggested several changes in the application and the 
application process that could contribute to a reduction in 
misreporting and misallocation of program resources. 

It is important to note, however, that the results of this 
error-prone mbdeling development effort are very preliminary. % School 
meal application requirements have changed since 1981 and new application 
procedures may elicit more accurate information. If verification 
requirements are implemented, they too, would alter misreporting 
patterns. These developments would probably outdate the specific model 
generated by this study . Therefore, ^ the model developed in this analysis 
should not be implemented until it is tested with more recent* IVPP data. 
However, the analysis performed in developing this model revealed 
.important information about the nature of misreporting. It appears 
reasonable to use this information to improve the meal application 
process. ^ . 

Evidence from the analysis suggests that changes to the application 
could reduce misreporting. First, th6 analysis strongly suggests that 
applicants must report detailed income information for each adult. This 
would be accomplished by providmg separate lines for each - adult 
household member to report income from each source (e.g., wages, 
pensions, welfare, etc.), and requiring that a response be placed on each 
line (including zero or none if no income is received from that source). 
Income may be underreported bn the application because prior and current 



EXHIBIT 4: GROUPS WITH ABOVE AVERAGE PROPORTION OF ERRORS . 

- -:. . ■ H ' r-^ ■ — - — ' \ — ■ : — 

* * Percent of ' '\ 

.Group' ' Sample J ErrorVRate- 



Participants not receiving Food Stamps; 4.4 . S^.;! 

female householder absent or in the . 

non-farm .labor force, male householder in ' ^. / ; 

noh-.farm labor force; near a large city. ^ ,v 

Tar^ticipants not receiving Food Stamps; . * 2.5 62.9 

female householder "absent or in non-farm 
labor force; male householder in non-farm 

labor force; not neap a large city; and _ . 

no eligible siblings or at least one \ 
sibVing pays. 

Participants not receiving Food Stamps; 6.2 49,0 

female householder absent or in non-farm \ 
labor force; male householder in nonrfarm . . . 

labor force; not near a large city; : , , 

siblings, attend school , but none pays for 
lunch; blacK, Hispanic or other. 

Participants not rkeiving Food Stamps; 3.5 . 45.4 

female householder absent or in non-farm, 

labor force; male householder in non-farm. 

labor force; siblings, attend school , but 

none pay for lunch; white or Asian;- small. ./ 

town ior near medium sized city. ^ 

Participants not receiving Food Stamps; 8.2 40.4 

female absent or in non-farm labor force; . 
male absent, farms, or is not in labor ' 
force; female completed: high school .. , 

Participant not receiving Food Stamps; 4.1 23.8 

female absent or in the non-farm labor 

force; male absent, farms, or is not in the 

labor force; female^ did not complete " 

high school . . 

Participant not receiving, Food Stamps; 3.8 41.5 

female farms or is not in labor force; ' 
male in non-farm labor force; no 
siblings attend schools, or at least 
one sibling pays. 
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EXHIBIT 4: GROUPS WITH ABOVE AVERAGE PROPORTION OF ERRORS (Continued) 



' , ■ ■ Percent of 

GroufJ ' ' Sample Error Rate 

Participants receive no Food Stamps; . . 12.8 ' 24,8- 

female f^rms or Is not In labor force; ' 
male in non-farm labor force; some 
siblings 1n school and all participate. 

Participants receive Food Stamps and 4.1 29,4 , 

both male and female householders are in 
non^farm labor force. 
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EXHIBIT 5: , GROUPS, WITH 'BELOW AVERAQEi;ERROR 'RATES 




Group 



Percent of 
Sample 



Error R 



Participants not recelvlng^Food Stamps; f 
female householder absent or. Is In' non-farm 
1 abor force ; male househol der 1 s 1 n 'I ' 

, non-farm labor force; rural area/not near 
city; some siblings In school but all pay; 

. white or Aslap. ^ 

^ I .' f ^ ^ ■i,/ ... ■ . 

Participants receive no Food Stamps; , 
female farms or Is not In labor force; 
mal e absent , farms , or 1 s not/ 1 n 1 abor 
force;, students In grades 4-6 or j:Q-12^j 

Participants do not receive Food Stamps; 
female .farms or' Is not. In labor foi'ce; 
student In grades 1-3 or 7-9. 

Participants receive Food Stamps; 
.male 1s In, non-farm labor force; . ; 
female 'Is absent, farms or; Is not 1n 

labor force. 

Participants receive Food Stamps; 
male, householder Is absent,'' farms, 
or Is not In labor force. 
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(to ii lesser degree) appllcatlona required the fippllcqnt to aggregate 
Income for all adults. This matKod of requesting income Information did 
not atlmulate the respondent to consider all sources of Income for all 
adults, In contrast, the NESNP Income data and the In-home audits 
(discussed in the next chapter) quarjed resppridants about each Income 
source, for each adult, The error-fjronia analysis findings, which .showed 
the presence of multiple wage earners and multiple Income sources to be 
indicative of misreporting, provide evidence ta support this 
recommendation, ' 

Two other changes could be helpful. One that is * suggested by this 
analysis (and which -Was implemented by FNS in 1982-83), is requesting 
respondents to indicate whether they are receiving Food Stamp benefits. 
In. addition to being useful for selective case actions/ this question 
will also convey to applicants that a complete pljcture of the family 
resources is being obtained, which may help to deter misreporting. \ An 
additional general change to the application process is suggested by the 
error-prone analyses. If school districts require families to submit 
separate applications for each child, efforts should be made to 
* cross-check , the ; applications. A significant percent of families in -the, 
NESNP survey reported that children from the same household had "different 
eligibility statuses for school meal benefits. 



END NQTBS 



^Thi« type of data waa to be daveloped «s a part of IVPP. EPM 
development using IVPP, audit data la presented In Chapter 3 of this 
report. 

■ , ■ , ■ 

. U , ■ ( • ■ 

. • ■ .' . ; ■ ' . * ' ' 

'Although previously allowed, hardship deductions are no longer 
considered Jn eligibility determination. i, 
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REFINEMENT OF THE ERROR PRONE PROFILE BASED ON 
. 198-1-82 IVPP DATA 

Although 'the analyaU of tha NESNP clatd did succaasfMlly produce 
an. errop-prone modaU It had two major deflclanolea. FIrat/ the beat 
predictors of arrora were baaed on a combination of var^lablos^ some of 
which are on the application ' and^ others which are not but are probably 
accessible, Use of the NES'NP' model would require time consuming 
screening on the part of school staff. A second deficiency in the NESNP 
model is that the definition of an erroneous application was based 6n^ 
self-reported data, not data verified in an audit, Data frOm the 1981-82 
Income Verification Pilot Project were used to continue exploring the 
development of an error prone model. The IVPP-provided audited 
application data were used to answer the following questions: 

1, Did the findings from the preliminary NESNP-based model ». ' 

replicate using the IVPP in-home audit data? 

2. Gould the NESNP model be refined/ or a new model' developed, 
that would be effective, yet very easy to use? 

. The remainder of titis chapter presents the responses to these 
questions. The following topics are covered: 

• The 1981^82 IVPP Sample, \ ' « 
\ • Statistical Methods and Results, and 

• Conclusions. . 

The 1981-82 IVPP Sample / / / '^^ 

Because of the exploratory nature of the IVPP eff9rt;vin 1981, the 
sample of schpol food authorities was not drawn randoml^'. .^^ther/ an 
attempt was made "to obtain variation among districts in t^^^pf >ize/ 
location and urbanicity. Consequently, the statistical>-g^^rall'3|i^^ 



of flndlngfi from 108^82 I VP P may be questlonecl. ^ H^aili^, iW findinaa 
frpm previous analy^ses^ .Unded to la's Qon|tVt#n^ I fteross SFAi, 
suggesting a lack of 'strong Was and Implylnr- that findings are 

probably sogiewhat generallstable, ' 

. ■, '''''' ' . ■ -,4 *r ■ , . 
The arror-prone model analysis used a random sample of 741 .f ami lifts' 

from 54 schools In nine SFAs that participated jli-f hase I of IVPP, These 

741 families 'purtlelpated In; personal 4nl:e^rvl«w^; during which they were 

asked to proVldo ' documents (e.g./ fjay' stubs, letters from employers ^ 

etc";); that Indicated the. amount of Iricoi^e received from 17 sources (e.g., 

wages, ponsfonsV alimony, ate.) duHhg the^-^^m^^ In which -they applied 

for benefits. In addition, houseiibid size War reviewed. Thesa, In-home 

audits, ^theViafore, produced venlfiad Information from which ellgiblllty, 

could be computed/ and V compared with , the allglbility baaed on the 

information reported .on the/^(3H?i||cation for meal benefits, Cases of 

excess benefits , occurred when In-home audit data ' yielded a lower 

eligibility rating tharr did the-a|!)plicatlon data; 

statistical Methods and Resutts - Verification of NESNP Mod el 

, The analysis of tlie iVPP in-Kdipe audit data first addressed the 
validation of the NESNP fiiidings'. Each respondent in the in-home audit* 
sample was assigned a score (log , of the, odds ratio) reflecting (^the 
likelihood of receipt of excess: benefits as predicted by the NESNP 
model. The score was' then correlated with Whtfther or not the applicant 
had actually received; .excess benefits. This resulted in an index of 
discriminant power/ Jhe^ was 0:46 for the original NESNP 

profile. in compar.isprfi* when i application month data from the in-home 
audit were used, a ;cJorreia^tion of 0.36 was obtained. A second 
correlation was prod6ced using in-home audit month data. This produced a 
correlation of 0.48- The second correlation is very close to that from 
the NESNP data. Therefore; the profile as a^ whole replicates findings 
from the in-home audit data base. This important point should be noted: 
tlie correlationV prodgced with in-home audit month income and family size 
data are much SjtrQnger^ t^ those produced with application month data. 
This result^rrob^^^^ both the NESNP survey data and the 

in-home lui^lt mdhth income data were collected at the same time--durmg 



8prln9«-sbQut nin© inenths s^lir the applications had ;been submitted- 
to sohool offlol'sis f9r approval, ThiPefope, the NESNP eppor-ppone^ 
ppoflle PifliBti 9 6ompoundlng of th§ §ffmU of Jnltial miipepoptlng on 
ma^l b#niflt applleatlons and eligibility, itatus ahanga duPingi the coupi©. 
of tlv© school year. 

Examination" of eppor patea fop the vaploiis eppop'-prone^ gpoups 
(that la, those ppoupa with a atpon^ likelihood of peeeiving' enfieas 
benaflta) must be apppoaohed eaufclooaly beeauae the In-home audit sample 
aixe la too small to ppopoply vepify the smailep splits (groupa) In the 
modal. Hovyevep, the prlmapy splits pan be vyplfled. Exhibit O ppeaenta 
"the pfl«iilt«(of the validation of the ppoiips. Each bar peppesenta the 
propoptlon of each gpoup pecelvlnp axcess benefits. The two pplmapy 
apllta wore 'rocolpt of Food Stomps and female ndult houaohold membap |n 
the labor force. The validation analysis of the NF.SNP orroiJ^pPone 
profile has an Important implication for Intorprotlng findings from the 
in-liome audits. -The unrepresentative sample of SFAs ahd schools from 
which the in-home audit sj^mple was drawn Raised questionsi as to the 
generalizability of findings from the sample, The NESNP Uample, by 
contrast, Is nationally representatrve and was constructed in accord with 
generally acceptgd"^ statistical principles. Therefore, the fact that the 
two surveys replicate on prinoipal correlates of receipt of excess meal 
program benefits suggests that other finding's from the ih-home audits may 
also replicate nationally, ^ 

Statistical Methods and Results - An Improved Model Based on IVPP Data 

The next issue addressed by the analysis^ was whether or not an EPM 
could be produced that would not be very ^burdensome for an SFA. to use. 
It was decided that a model would be reasonable for implementation if 
it: 

• relied on the most accessible informsition; that is, the 

information reported on the application; and 

• involved very few "procedural or screening steps. 

Therefore, this effort excluded many of the variables (e.g., nonfarm 
labor force, sibling's participation status) that were effective 
predictors in the NESNP model EPM becaiise they were not currently 
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Participants not rtctlving food stamps; f«iia1« hous^holdar 
absant' or in tha non.farm labor forca; (na1« housaholdar In' 
non.farm tabor forca; naar a larga city. 

Participants not racalving food stamps; fana1« housaholdar 
absant or in non-farm labor forcaj mala housaholdar In 
non.famt labor forca* not naar a iarg« city; and no 
a11g1bl0 siblings or at Itast on« sibling pays. 

Participants not recalving food stamps; ftmalt housaholdar . 
absant or In non.farm labor forca; mala housaholdar In * 
non-fahn labor forca; not n«ar a larga city; siblings 
attand school, but nona pays for lunch; black, Hispanic, or 
othar, \ ■ 

Participants not racalvlpg food sHmos; famala housaholdar 
absant or In non-farm labor forca: mala housaholdar In 
non-farm labor forca; siblings attand school, but nona pays 
for lunch; vnhlta or Asian; small town or naar (nad1um*s1zad 
city. 

Participants not racfivln^ food stamps; famala housaholdar 
absant or in non.farm labor forca; mala housaholdar In 
non^farm Ubor ^orca; rurrl araa not naar city; soma 
siblings In school but all pay; whita or Asian. 

Pantlclpants not racalving food stamps; famala absant or In 
non-fann labor forca; mala atfsant, farms/ or Is not In 
labor force; female completed high school. 
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Participants not>acelv1ng food stamps; female absant or In 
tha norv-farm labor forca; mala absent, fanas, or Is not In 
tha labor *forcet female did not complete high school. 

Participants not receiving foo<l stamps; 'female farms or is 
not In labor force; male \n non-farm labor force; no 
siblings attand school, or at leist one sibling pays, > 

Participants receive no food stamps; female farms or Is not 
In labor forca; male in non-farm labor force; some siblings 
In school and all participate. 

Participants receive no food stamps; fwijile farms or Is not 
In labor force; male absent; farms, or is not In labor 
force; students In grades 4-6 or lO-U- . 

Participants do not receive food stamps; female farms or Is 
not In labor force; mal* absAnt; student In grades 1-3 or 
7-9. 

Participants receive fooo stamps and both male and female 
householders are in non-farm labor force. 

Participants receive food stamps; the male Is In the 
non-farm labor force; tha female Is absent, farms. Or is, 
not In the labor force. ^ 

Participants receive food stamps; the male householder 1$ 
absent, farms, or Is not in the labor force. 



T^tal 
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requested or anticipated to be requested on the applicatioa. The 
variables that are available on the application and were tested as' 
potential predicto^r? are shown in Exhibit 7. , 

ft 

' The only application predictor of receipt ^ of excess benefits that 
replicated in all nine sampled SFAs was what we call **threshold 
reporting." "Threshold reporting" is reporting a monthly income that is 
on or near the free, or reduced-price eligibility cut-off point. For 
example,, a family of five is eligible for reduced-price benefits if it 
has a monthly income of $1,516 or less. If a family of five reported its 
monthly income to be $1,506, it would, under our definition, be a 
"threshold reporter." Exhibit 8 displays the relative frequency of 
receipt of excess benefits by difference between eligibility cut-off 
point for benefits received and reported income. Seventy percent of 
those who reported incomes within $60 a month of 4th e eligibility ) line 
receive excess benefits compared with only 10 percent of those who 
reported incomes more than $180 a month from, the eligibility cut-off. 
(The $60 intervals used are not arbitrary but were constructed to 
maximize the discriminant power and robustness of differences.) 

The 1981-82 application form used in this study did not contain 
information on receipt of Food Stamps. However^ the FNS school meal 
benefit application recommended for the 1982-83 school year does request 
this information. Therefore; Jt was appropriate to examine receipt of 
Food Stamps relative to receipt 6f excess benefits, even though Food 
Stamp participation could only be identified during the in-home audit. 
Food Stamp ''participation data-^'can be used to improve significantly the 
prediction of receipt of excess benefit reporting. Use of Food Stfanp 
participation data in the error-prone model rests on the assumption that 
such information ^will not be significantly thisreported on meal benefit 
applications! T0\s assumption may be reasonable on the grounds that 
applicants for school meal benefits have no overt motivation to misreport 
food stamp participation. Eligibility for school meals is based solely 
on income and family size. It is -possible, however, that some applicants 
incorrectly believe that Food V^^^^ participation does' affect eligibility 
and therefore, may not report it on the application. 

• ■ ■ 

(? ■ • • 
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EXHIBIT 7: LIST OF VARIABLES ^USED IN. DEVELOPMENT OF APPLICATION-BASED 
ERROR-PRONE PROFILE 



Income from wages and other earnings. 

Income from Sodal Security 

Income from public assistance ^ 

Income from unemployment benefits 

Income from child support or alimony 

Income from pension or retirement 

Other Income 

Family size 

Total reported Income 

Eligibility status based on application 

Difference between reported Income and eligibnity cut-off point 
Whether income was reported in round numbers (evenly divi sible by 25, 50 
Presence, or absence of Social Security numbers 
Grade of student 
Receipt of Food Stamps* 
Eligibility based on in-home audit* 

Difference between application and in-home audit based eligibility* 



*Data.from in-home audit used. All other variables from application. 
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EXHIBIT 8: PERCENTAGE OF HOUSEHOLDS RECEIVING EXCESS BENEFITS BY DIFFERENCE 
6F REPORTED MONTHLY INCOME FROM ELIGIBILITY THRESHOLD 
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' Food Stamp participation status was added'.^): to the application 
variables'' shown in Exhibit 7 and a new error-pronW^'i^odel was developed 
through a six-stage process. First, bivariate pislationships between 
excess benefits and application variables were examined and the 
application variableis were reseated to maximize predictive power. Second, 
^a step-wise logistic discriminant function model was^' estimated , to select 
•the most predictive set of variables. Third, .a sequential search using, 
automatic interaction detection (AID) was conducted >to assure that no 
hidden interactions existed among the variables that were not discovered 
by previous steps.. Fourth, a prediction model was estimated using a 
weighted :#least-squares procedure that ^ optimized discrimiriatit power. 
Fifth, ^he prediction equation was simplified ^ to allow easy uset by SFA 
officials. Sixth, the prediction eq\jation was tested sep'aratiely for each 

^ ■ 

of the nine sampled SFAs, 

The . resulting error-prone model is very; simple and contains only 
two variables. Exhibit 9 presents the scoring profile derived from the 
model. An application is given one point if it reports an income within 
$120 a month of the free or reduced-price eligibility cut-off . Another 
point is added if the reported 'income is within $60 of the free or 
reduced-price eligibility cut-off. Finally, a point is subtracted if the 
applicant reports receiving Food Stamps. The resulting scale has values 
ranging from -1 to +2. Exhibit 10 displays the percentage of all sampled 
applicants receiving excess benefits by error-prone score and the 
percentage of the sample in each of the score categories. In each of the 
nine sampled SFAs, the error-prone model significantly discriminated 
between applicants receiving excess benefits and all other applicants. 

In addition to being easy to. use, ' this model has the following) 
beneficial features: 

• It is able to separate nearly half the applicants into a 

group of accurate reporters (45% of tjine sample received Food 
Stamps and only 2% of these Food Stamp recipients received 
excess meal benefits) . Identification of accurate reporters is 
helpful because it prevents the waste of pursuing verification 
of these applications. 

• The 6 percent of applicants who receive an error-prone 

score of "2'* have applications that are four times as likely to 
contain an error that would reduce eligibility status as an 



EXhilBIT 9: APPLICATION-BASED ERROR-PRONE PROFILE SCORING SYSTEM 



Step 

A. . If reported income Is within (A) 

S120 a month of the free or reduced- 
price eligibility tut*-off, write *1V on . 
line A. Otherwise write *0. * 

B. If reported income is within $60 (B) 

a month of the free or reduced- 
price eligibility cut-^off, write on 

line B. Otherwise write '0.' 

-■■ . ^ . ■ 

C. If the applicant reports receiving (C) 

Food Stampsr, write on line C. 

Otherwise write '0. ' , 

Sum lines A, B, and C, and write^ . (D) 

final score on 1^'ne D. 
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EXHIBIT. 10: APPLICATION-BASED ERROR-PRONE PROFILE 



. ' ' . . ■ ■; 

• ... -f, • . ' • V 

Error-Prone "Percentage Receiving Percentage of 

, Score* "Excess Benefits Total Sample 



-1 


2% 




45% 




0 


20% 




42% 




■1. 


40% 


V 


7% 






71% 


6% 





* Error prone scores were derived statistically by a weighted lea^t 
squares procedure. A linear transformation was performed to preserve 
the relative weights, but produce whole numbers. 
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^ . applicant selected randomly from the total sample. 

■■■■ '-i ■ 

■'■■y^ The final step in developing the application-based error-prone 
model was to analyze what types of error are, and are not, detectable by 
the profile. The err;>r-prone model relies heavily on threshold income 
reporting. Therefore a particular concern in testing the model was to 
.determine whether the model detects only small reporting errors made by 
' households with actual incomes near their reported incomes. If this wer;e 
the case, then ^the error-prone model would fail to detect large errors 
\^ madp^b^ more affluent households. - 

; T'o addres'S this issue, misreporters identified by the scoring 

, Iprofile^ were compared with misreporters not identified by the profile in 

•rterms of how much greater their actual income was than the income cut-off 

T^fQr^the level of program benefits they received. The purpose of this 

^^comparison was to determine whether/the error-prone profile is biased 

^against Jdw-income households whose actual monthly incomes are close to 

tpd eligibility cut-off. Detectable misreporters had median monthly 

ilnlomes '4i230 above the maximum allowable income for benefits received. 

; 'i^his cbmpa'ces with a median monthly income of $255 a month above the 
- *^ ' ■ ■ . ' ' ' 

^maximum allowable irfcome received by nondetectable misreporters. 

■// * ■ 't ■ 

;> thirty-fbuLP percent of detectable misreporters had actual monthly incomes 
: Within > ^100 of the maximum allowable for benefits received compared with 
^ 30 percent of nondetectable misreporters. The difference of 4 percent 
'.jwas riot statistically significant. Therefore, the model does not appear 
to discriminate againsst those who are truly/ but marginally, eligible. 

SummalSy^hd Conclusions ^ ^ 

The 1981-82 IVPP in-home audit sample was Used to refine the 



^'r;f ^pfe^iminarv' model developed from NESNP data. The results showed that the 
V model^developed from the NESNP data base was validated on the in-home 
1^^. audit data. This result produces two important conclusions: 

• Results from a comprehensive review of income by source in a 
* ' personal interview that occurred in NESNP produced results 

^; similar to results from the in-home audit which required 

documentation to verify income. This finding strongly suggests 
that more Accurate income information would be obtained if the 
free-and reduced-price meal application requested income by 
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sou rceV to stimulate the applicant to consider air " 
income for each adult. / - 

■ ■ ■ • ' •'■ .> . I .• • 

• the NESNP sample was st§t1stijE 

IVPP was not. The replication o'f the NESNP findings on the IVPP 
^ data suggest that the IVPP results are probably generalizable 
beyond the sample of SFAs that parti cipated^---^^--^ 

The goal of producing a simple but effective EPM also was 
achieved. The scoring profile, which is easy for SFA officials to use, 
is Capable of\ selecting a subgroup of applicants for verification that 
have a 70 percent pi;obability of receiving excess benefits. The 
error-prone profile was successful in selecting applicants receiving 
excess benefits in all nine sample SFAs. Use of the errorrprone profile 
for a six percent or smaller sample would, on the average, Result in 
selection of applications for verification that have up to four times the 
probability of receiving excesss benefits as applications selected at 
random. - ' 

In 1982-83, IVPP is testihg more sophisticated verification 
procedures in a nationally representative sample* of SFAs, and will 
include "a large in-home audit sample. Application improvements 
implemented in 1982-83 may outdate the model, and the sophisticated 
verification techniques may alter misreporting patterns. Thus, it will 
be necessary to further examine the EPM developed in 1981-82 with the 
1982-83 IVPP data. 



END NOTES 



Mncome Verification PiloOroject, School Year\1981-82 In-Home 
Airditi^lndrngs7'~Appll6d-Manag6m6ht-S-clenc6s|^^ 
School Meal Program Participation; Applied Management Sciences, 
December 1982. 
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' APPENDIX A . 
PREDICTORS USED IN THE ANALYSES" 
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r Predictors - 

. » . ■ . '. >■-■••, 

1. Local density (city, suburb, small town, rural) 

2. Global density (near' large city, .nea> middle-sized city, 
small town, not near city) 

3. Number of children attending school 
-4T---Number-df'ch1Tdr^^^^ not attending sc 

5. Type of dwel liftg 

6". Location bf dwelling (acres surrounding home) 
?• Ethnicity of^target child ^ 
8. -Owns home 

'9. Highest grade attained by male householder 

10. Highest grade attained by female householder 

11. Work status of male householder (farm, non-farm, absent, 
or not in labor force) ^ ' 

^12. Work status of female householder:(:f^^^^ non-farm, absent, 
r or not in labor force) ^ 
13. Number of adults in household - 

>: ;15^ Participation of el igible' siblings (no eligible sibling, 

/" all participate, one, pays) 
:X^y; Voo6 Stamp participation 

'■■^'<-^^!f'. Receives food under WIC 

''^•''18. Receives welfare payments, . ; 

19. Relatlofiship of target child ta mother . 

20. Relationship of target cMl^ 
" 21 . Hi ghest gride attai ned byiel^thfer; househol der 

22. Participation in preylpiis/i^^ 

23. Number" of children I'S : 

24. Number of children 6-3!^;:;^^^^^^^^ 

25. Number of children U-ia;J;H0^^^^ / 

26. Number of unrelated adults; 

27. Number of related adults 
. 28. Income by wage-earner 

29. Self-employed, non-farm 

30. .Self-employed, farm / 

31. Income from dividends or interest 
.32. Social Security income 

33. Number of older siblings 

34. Number of younger, siblings 

35. Age of oldest sibling 

36. Age of youngest sibling 

37. Age of child ^ 
38. .Age of father 

39. Age of mother ' 

40. Occupation of father 

41. Occupation of mother i 

42. Sex of child ' ^ 



^Codes are* listed below. J 

1 = Appears in model 

2 = Used in AID run 

3 = Examined bivariately 
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Analytic Parameters of. the AID run were: 

One dichotomous criterion variable. 

^ — Eighteen predictors—seven free and ll^onotd'nic-- for 
■ the Food Stamp Model V 

~ Reducibility criterion = 0,002/ ^^v 

Minimum group si2e=40. 

To retain a spliti the phi coefficient (equivalent to the^ 
Pearson r) between the dichotomous split and the criterion variable in 
the cross-validation sample had to be large enough to have been 
significant for the working sample, and of similar directonality. 
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54< 



ERIC 



The foTlQw!ng define operationany the spl its produced by the AID 
analysis for the two models. For clarification of ^the predictor 
variables , NESNP documentation associated 'with the data col lection should 
be consulted. , *. - '*\ 

1. Male is in^^hon-f arm labor force : HQ128 is present and not equal to: 
9, 10, or 13, ■ ' . t . . ; 

2. Male is absent, farms, or is not in labor force : HQ128 is missing 
or equal to 9, 10, 13. 

3. Female is absent or is in non-farm labor force : ' HQ136 is missing or 
.is present and not equal to 9, 10, or 13; 

4 Female farms or -is not in labor force: HQ136 is equal to 9,. 10, or 

■ 13.;.;; / . ■ , . ; V; ^ ^ 

5. - Female is in non-farm labor force : HQ136 is present and not equal 

to . 9, 10, or 13. ' . . • \ 

6. Female is absent, farms, or , rs not in labor force : HQ136 is missing 
• or equal to 9, 10, or 13. ^ 

7. Near large or middle-sized city : HQ184 is 7 or less. ^ 

8. Small town or rural area, not near city : HQ184 is 8 or 9.' 

, 9 No eligible sit)linq or at least one sibling pays : None of HQ121-4 

to HQ121-12 is 1, or for one of the above mentioned variables the 

value is 1 and the value of the corresponding HQ123 variable is^|,. 

greater than 0 (or greater than 40 if reduced-price participant)-:-' 

in. All elioibVe siblings participate : At l^ast ^one of Hpll9-4to ^ 
HQ119-12 is' 1 and whenever this happens, the corresponding HQ1Z3 
variable is 0 (or not greater than 40 if reduced-price participant). 

11. Black or Hispanic : HQ180 is 2 or 3.. 

12. White or Other : HQ180 is 1 or 5. 

13. One or no siblings attend school : Number of HQ119-4 to HQ119-12 
equal to l is not grd'ater than one. 

14. Two or more siblings attend school : Number of HQ119-4 to HQl^a:^^^ 
equal to 1 is greater than one. 

. 15. Small town : HQ;^4 is 8. 
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16. Rural area, not near city ; HQ184 1s 9. ' - 

17. Male has not completed high school : HQ127 less than or equal. to 2 
or missing. , ' 

18. Male has completed high school : HQ127 gre,ater than 2. 

19. Cities, suburbs, or towns : HQ184 Is 1, 2. 3, 5, 6 or 8. . 

20. Rural areas :' Hqi84 is'4. 7. or 9. ' 

21. Owns house : HQ164 is 1. ■ . ^ . 

22. Does not own house : HQ164 Is not 1. 

;3. Receive no Food Stamps ; HQ147 is not 1 and HQ151 Is not 1. 
|24. Recei ve Food Stamp s : HQ147 ' i s 1 or HQ151 is'l. \^ 

25. Near large city V HQ184 is 1, 2, 3, or 4. 

26. Not near large city : HQ184 is 5, 6, 7, 8, or 9. 

27. Black, Hispanic, or other : HQ180 is 2, 3, or 5. ^ 
28^ White or Asian : HQ180 is 1 or 4. 

29. Small town orVnear middle-sized city : HQ184 is- 5, 6, 7, or 8. 

30. Temale completed high -^^ool : HQ135 is greater than 2. 

31. Female did not complete high school : HQ135 is 1 or 2 or missing. 

32. Grades 4-6 or 10-12 : GRADE is 4, 5, 6, 10, 11. or 12. ' ■ 

33. Grades 1-3 or 7-9: GRADE Is 1, 2, 3, 7, 8, or 9. 
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